Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

single precision support in skimage.restoration module #5219

Merged
merged 14 commits into from
May 21, 2021

Conversation

grlee77
Copy link
Contributor

@grlee77 grlee77 commented Feb 4, 2021

Description

This PR adds float32 support to wiener, unsupervised_wiener, and rolling_ball. Some tests for the expected dtype were added to other functions that already had float32 support.

Here I introduced a _float_dtype helper in _shared.utils to try to keep casting to floating point types consistent. Probably #5204 could use this as well.

Checklist

For reviewers

  • Check that the PR title is short, concise, and will make sense 1 year
    later.
  • Check that new functions are imported in corresponding __init__.py.
  • Check that new features, API changes, and deprecations are mentioned in
    doc/release/release_dev.rst.

@grlee77 grlee77 added the ⏩ type: Enhancement Improve existing features label Feb 4, 2021
@pep8speaks
Copy link

pep8speaks commented Feb 4, 2021

Hello @grlee77! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 46:29: E231 missing whitespace after ','

Comment last updated at 2021-05-21 14:32:11 UTC

Copy link
Contributor

@FirefoxMetzger FirefoxMetzger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work 👍

I've added a bunch of comments, but most of them are a repetition of what I commented earlier, just to put it in every place that I found that may need a change.

In general, I wonder if functions should return the same dtype as the input image, or if they should return the dtype in which the processing was done. E.g., if the input is FP16, should it return FP32 or FP16?

skimage/_shared/utils.py Outdated Show resolved Hide resolved
skimage/_shared/utils.py Outdated Show resolved Hide resolved
skimage/restoration/_denoise.py Outdated Show resolved Hide resolved
skimage/restoration/tests/test_denoise.py Outdated Show resolved Hide resolved
skimage/restoration/tests/test_inpaint.py Outdated Show resolved Hide resolved
skimage/restoration/tests/test_rolling_ball.py Outdated Show resolved Hide resolved
img = 155 * np.ones((100, 100), dtype=dtype)
kernel = ellipsoid_kernel((25, 53), 50).astype(dtype, copy=False)
background = rolling_ball(img, kernel=kernel)
assert background.dtype == img.dtype
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will always be true, because we cast background to img.dtype after applying the kernel. The question is if we want that or if it should be background.dtype == np.float32 if img is F16.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have left the dtype behavior in rolling_ball as-is for now. I think this was mainly as a convenience for users providing integer valued inputs to get integer valued outputs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We introduced this because of integer images - I never wrote an optimized integer kernel -, but it affects all inputs equally. The general idea was "dtype in, dtype out".

I think it will be best if the behavior here regarding the return type is consistent with the other functions in the library. If that means changing the current behavior, I think it makes sense to do.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this concern is orthogonal to the purpose if this PR, so let's just defer this decision so as not to hold the PR up unnecessarily.

skimage/restoration/tests/test_rolling_ball.py Outdated Show resolved Hide resolved
skimage/restoration/uft.py Outdated Show resolved Hide resolved
skimage/restoration/uft.py Outdated Show resolved Hide resolved
skimage/_shared/utils.py Outdated Show resolved Hide resolved
@grlee77
Copy link
Contributor Author

grlee77 commented Feb 10, 2021

Here is a proposed _support_float_type implementation based on the reviewer suggestions. I added two features that I would like feedback on. I moved

1.) optionally raise ValueError if the input is complex-valued
2.) handle a sequence of dtypes for use in functions with more than one relevant input (e.g. skimage.metrics functions that deal with comparing two images).

I added float64 and float128 to the new_float_type dict for clarity, even though it is not strictly necessary since float64 is the default when a key is not found.

new_float_type = {
    # preserved types
    np.float32().dtype.char : np.float32,
    np.float64().dtype.char : np.float64,
    np.complex64().dtype.char : np.complex64,
    np.complex128().dtype.char : np.complex128,
    # altered types
    np.float16().dtype.char : np.float32,
    'g' : np.float64      # np.float128 ; doesn't exist on windows
    'G' : np.complex128   # np.complex256 ; doesn't exist on windows
}


def _supported_float_type(dtype, allow_complex=False):
    """Return an appropriate floating-point dtype for a given dtype.

    float32, float64, complex64, complex128 are preserved.
    float16 is promoted to float32.
    complex256 is demoted to complex128.
    Other types are cast to float64.

    Paramters
    ---------
    dtype : np.dtype or Iterable of np.dtype
        The input dtype. If a sequence of multiple dtypes is provided, each
        dtype is first converted to a supported floating point type and the
        final dtype is then determined by applying `np.result_type` on the
        sequence of supported floating point types.
    allow_complex : bool, optional
        If False, raise a ValueError on complex-valued inputs.

    Retruns
    -------
    float_type : dtype
        Floating-point dtype for the image.
    """
    if isinstance(dtype, Iterable) and not isinstance(dtype, str):
        return np.result_type(*(_supported_float_type(d) for d in dtype))
    dtype = np.dtype(dtype)
    if not allow_complex and dtype.kind == 'c':
        raise ValueError("complex valued input is not supported")
    return new_float_type.get(dtype.char, np.float64)

@grlee77 grlee77 added this to In progress in skimage2 API Feb 15, 2021
Base automatically changed from master to main February 18, 2021 18:23
grlee77 added a commit to grlee77/scikit-image that referenced this pull request Apr 13, 2021
grlee77 added a commit to grlee77/scikit-image that referenced this pull request Apr 16, 2021
grlee77 added a commit to grlee77/scikit-image that referenced this pull request Apr 21, 2021
Copy link
Member

@jni jni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @grlee77! I've only made a minor and super optional code style comment.

@FirefoxMetzger identified a test that is possibly-not-comprehensive-enough here, and I also wonder whether you've overlooked that?

skimage/restoration/deconvolution.py Outdated Show resolved Hide resolved
img = 155 * np.ones((100, 100), dtype=dtype)
kernel = ellipsoid_kernel((25, 53), 50).astype(dtype, copy=False)
background = rolling_ball(img, kernel=kernel)
assert background.dtype == img.dtype
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this concern is orthogonal to the purpose if this PR, so let's just defer this decision so as not to hold the PR up unnecessarily.

@jni
Copy link
Member

jni commented Apr 26, 2021

(But note that I'd be happy to merge this as-is in the interest of keeping momentum going with this series of PRs.)

@jni
Copy link
Member

jni commented May 12, 2021

@grlee77 do you want to pick this one up again?

- use a test image with more structure
- fix the random seed
- tune denoising parameters a bit

With this updates the difference between 3D vs 2D is more pronounced.

test float16 dtype
@grlee77
Copy link
Contributor Author

grlee77 commented May 13, 2021

@grlee77 do you want to pick this one up again?

Sure, I have addressed the initial comments and rebased. I updated one of the non-local means test cases, which happened to give almost the same PSNR for the two cases being compared so that it has a more suitable test object than a simple square.

Copy link
Member

@rfezzani rfezzani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @grlee77, that's amazing! I just left really minor comments, otherwise it is ready to merge 😉

skimage/restoration/_denoise.py Outdated Show resolved Hide resolved
skimage/restoration/deconvolution.py Outdated Show resolved Hide resolved
skimage/restoration/tests/test_restoration.py Outdated Show resolved Hide resolved
skimage/restoration/tests/test_rolling_ball.py Outdated Show resolved Hide resolved
skimage/restoration/uft.py Outdated Show resolved Hide resolved
@rfezzani rfezzani merged commit 00875e6 into scikit-image:main May 21, 2021
@rfezzani
Copy link
Member

Thank you @grlee77 🎉

@grlee77
Copy link
Contributor Author

grlee77 commented May 21, 2021

Thanks @rfezzani, I have implemented your suggestions and CI is green.

@grlee77
Copy link
Contributor Author

grlee77 commented May 21, 2021

Nevermind, I see you merged just before I added the comment! Thanks for the detailed review.

@grlee77 grlee77 moved this from In progress to Done in skimage2 API Jun 21, 2021
@grlee77 grlee77 deleted the float32_restoration branch July 8, 2021 20:45
@imagesc-bot
Copy link

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/wiener-deconvolution-implementation-handling-complex-psf-in-scikit-image/94170/3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
⏩ type: Enhancement Improve existing features
Projects
Development

Successfully merging this pull request may close these issues.

None yet

6 participants